Researchers found that Vision Transformers (ViTs) learn Gestalt-like figure-ground cues such as surroundedness and convexity from natural images, with mixed results for symmetry. The study positions V…
Researchers introduced HetDPT, a depth pruning method for Vision Transformers that speeds up DeiT-B by 1.58x and DeiT-S by 1.39x with minimal accuracy loss, and combined with width pruning (HetDPT+) a…
Researchers introduced FlexViT, a reconfigurable FPGA accelerator that achieves up to 2.74x speedup for Vision Transformer layers on edge devices. The accelerator uses a hardware-software co-design ap…
Researchers introduce Depth-wise Gradient Augmentation, a new optimization paradigm that transforms layer-wise updates to exploit structured relationships across deep neural network layers. Their inst…
Researchers have developed DiffusionBlocks, a framework that partitions transformer neural networks into independently trainable blocks to reduce memory requirements proportionally while maintaining c…